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ABSTRACT 

Findings of a study that examined the relationship 
between aspiring educational administrators* graduate school 
admissions test performances and their assessed skill dimensions are 
presented in this paper. Specifically, the study sought to determine 
the relationship between graduate admissions test scores, as measured 
by the Miller Analogies Test (MAT) and the Graduate Record 
Examination (GRE) , and the applicants' skill dimensions, as measured 
by the Assessment Center Project of the National Association of 
Secondary School Principals (NASSP) . The assessment enter obtained 
samples of participants' job-related behaviors to predict their 
administrative aptitudes. Methodology involved analysis of the 
admissions test and assessment scores of 121 participants. Findings 
indicate that admissions test score did not reflect the applicants' 
job-related behaviors, though those with high admissions test scores 
were almost twice as likely to receive high on-the-job ratings. The 
recommendation is made to develop a standardized, national graduate 
entrance test to assess analytic ability and administrative aptitude. 
One table is included. (Contains 25 references.) (LMl) 
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Do Graduate School Admissions Tests Relate 
to Assessment Center Dimensions? 
The leadership required of building administrators 
is well documented in the research on effective schools* 
Confirmation of the importance of principals has 
fostered an examination of their preparation (Leithwood, 
1984), selection (Zakariya, 1983), and role ( Principals 
for Our Changing Schools , 1990) . Educators have called 
for a review of preparation programs for principals 
(Performance-Based Preparation. 1985; Leaders for 
America' s Schools , 1987) , developed alternate and 
promising selection practices (Baltzell & Dentler, 
1983)/ and sought ways to improve those programs and 
practices (Cornett, 1983; Manasse, 1983; & Shilling, 
1986) . 

The National Association of Elementary School 
Principals and the National Association of Secondary 
School Principals jointly sponsored the National 
Commission for the Principalship to explore development 
of a national certification process. The members of the 
National Commission have called for revamping of 
traditional preparation and state certification 
programs . 

The two Associations agree that preparation 
programs should reflect primarily the real world of 
the operating school. Earlier theory-based 
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constructs have become largely obsolescent, 
overcome by the flood of change engulfing schools • 
In their place new programs reflecting inductive 
analysis, realities of the workplace, and attention 
to the programs and operations of schools must be 
designed (Thomson, 1990, p. 1) . 

Principals are key players in the success of their 
schools, as Sweeney (1982) reported in a synthesis of 
the research on effective schools. In eight studies, 
Sweeney found a positive correlation between leadership 
behavior and school outcomes. School effectiveness was 
increased by principals who emphasized achievement, set 
instructional strategies, provided an orderly school 
atmosphere, frequently evaluated student progress, 
coordinated instruction, and supported teachers. These 
findings corroborate the importance of valid selection 
procedures for building administrators at the university 
admissions level, as well as at time of employment. 

Members of the National 'Policy Board of Educational 
Administration (1989) proposed the following sweeping 
changes in administrator preparation programs: 

The National Policy Board recommends that entrance 
standards to administrator preparation programs be 
dramatically raised to ensure that all candidates 
possess strong analytic ability, high 
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administrative potential, and demonstrated success 

in teaching including: 

Assessment of analytic ability and 
administrative aptitude by a standardized 
national test, with admission to preparation 
programs limited to individuals scoring in the 
top guar tile, and 

^ Assessment of teaching excellence by state 
licensure, a master's degree in teaching, and 
evidence of successful teaching in a classroom 
setting. 

Administrator preparation programs must deal 
forcefully with the problems of low-quality 
applicants and lax selection by dramatically 
raising entrance standards. Only those individuals 
with strong analytic ability and demonstrated 
excellence in teaching should be admitted to 
administrator preparation. 

Toward this end, we (The National Policy 
Board) propose that a standard national examination 
be developed for prospective educational 
administrators, similar to those required for 
admission to other professional programs (MCAT, 
GMAT, or LSAT) . The examination will evaluate 
candidates' verbal, mathematical and reasoning 
abilities, as well as other skills indicative of 
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administrative potential. Only those candidates 
who score well above average--the top quartile is a 
reasonable cutoff — will be eligible for admission. 
( Improving Preparation q£ ££lml 
Administrators , 1989, p. 14). 
Traditionally, students pursuing a career in 
administration are required to have at least two years 
of teaching experience (Burks, 1988) . The requirement 
of teaching experience, however, does not necessarily 
lead to selective admissions. 

Many graduate programs adhere to an unspoken pact 
that any teacher, even an unsuccessful teacher with 
marginal academic ability, has an inalienable right 
to study for an administrator's certificate, and 
persistent candidates are almost always admitted. 
The end result is a glut of certificate holders 
with dubious qualifications who cannot find school 
administrative jobs. ( Improving the Preparation ^ 
School Administrators , 1989, p. 10). 
For admission to a graduate program, students are 
often required to take a standardized entrance 
examination, such as the Miller Analogies Test (MAT) or 
the Graduate Record Examination (GRE) . Results from 
these tests are weighted differently, depending upon the 
institution and department (Oltman & Hartnett, 1985) . 
Nor are admissions standards rigorously administered, as 
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the average GRE scores of educational administration 
students in 1985-86 ranked ninety- first among 94 
intended majors of graduate students (Griffiths, Stout, 
& Forsyth, 1988, p. 290). Additional requirements, that 
vary from institution to institution, are likely to 
include the submission of transcripts of previous 
coursework, autobiographical sketches, statements of 
career goals, letters of recommendation, and samples of 
writing. 

Upon completion of an advanced degree, prospective 
administrators compece for administrative positions. 
Applicants are generally required by hiring officials to 
submit letters of application, resumes, placement 
papers, and completed application forms. Hiring 
officials may also require applicants to submit 
transcripts, complete pencil-and -paper tests and 
inventories, write position papers on school-related 
topics, engage in interviews--sometimes structured ones, 
or complete an internship (Baltzell & Dentler, 1983). 

Information about candidates for graduate school 
and administrative jobs can be gathered by means other 
than paper-and-pencil tests, interviews, and traditional 
paper-screening measures. In addition to the use of 
graduate school admissions examinations such as the 
Miller Analogies Test (MAT) and the Graduate Record 
Examination (GRE) , participation in an assessment center 
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may provide helpful information- In some states, 
applicants for initial administrative certificates must 
participate in an assessic:ent center (Wendel, Yusten, & 
Gappa, 1988) • 

Paper-and-pencil tests provide "signs" of 
individuals' abilities whereas simulation • exercises 
extract "samples" of actual job-related behaviors. 
Persons who score highly on a test of verbal ability may 
or may not be good writers or speakers, but their skills 
in writing and speaking can be authentically assessed by 
having them write and speak in simulations, 

Purpose 

The purpose of this study was to determine the 
relationship between (a) graduate admissions test scores 
from the Miller Analogies Test (MAT) and the verbal and 
quantitative scores of the Graduate Record Examination 
(GRE) and (b) skill dimensions as measured by the 
Assessment Center project of the National Association of 
Secondary School Principals, the leading assessment 
center program for principals- Participants in the 
study completed one or both of the graduate admissions 
tests and participated in an assessment center after 
admission to a graduate degree program in Educational 
Administration, 
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Background and Context 
The MAT ^n4 the GRE 

For entry into a post-baccalaureate program, 
students are frequently required to take an admissions 
test. In the field of education, generally either the 
Miller Analogies Test (MAT) or the Graduate Record 
Examination (GRE) is required. The I^T is a test of 
verbal reasoning, and the GRE is an aptitude test. Both 
are presumed to be predictors of academic success. 

Tlie Miller Analoaie? Test . The MAT is a 100 item 
test of information and verbal reasoning ability. The 
analogies use information from a number of subject areas 
including contemporary culture, the humanities, social 
sciences, natural sciences, mathematics, and 
linguistics. Reasoning ability is tested through 
recognition of the relationships among the terms of an 
analogy (Cash, 1979) . 

lil^ graduate Record Examine t ion ■ The GRE is an 
aptitude test given several times a year by The 
Educational Testing Service of Princeton, New Jersey. 
The bajic GRE takes 3 1/2 hours to complete. The 
reliability coefficient for Education is .93 (GRE Guide, 
1974) - The GRE consists of multiple choice questions on 
numerous topics. The resulting scores are divided into 
two areas--verbal and quantitative. The verbal portion 
covers verbal reasoning and reading comprehension. The 
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GRE verbal ability scores have -tended to be more valid 
in such verbally oriented disciplines as English and 
education than in scientific fields" (GRE Guide, 1974, 
p; 13). In the quantitative section, there are 
questions over arithmetic reasoning, basic algebra, 
geometrs-^, and the interpretation of data. An overall 
score is also given. 

The predictive validity of these tests for academic 
and career success is debated (Thornell & McCoy, 1985) . 
Payne, Wells and Clarke (1971) contended that the GRE 
scores, with a range of validity coefficients between 
.08 and 0.47, were rather unpredictable. Thacker and 
Williams (1974) did a review of literature on the GRE 
and concluded that the validity of the GRE to predict 
success in graduate students was very doubtful. 

Oltman and Kartnett (1985) scudied the use of the 
Graduate Record Examinations in programs across the 
country to determine if there was any discernible trend 
in the usage of the GREs. They found great variability 
in requirement for the exams and also differences in the 
importance given to the scores. In most departments, 
test scores from the GRE ranked below undergraduate 
performance and recommendations in determining 
admissions. Many departments reported that they used 
the test scores as compensation for otherwise weak 
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credentials. A second use was non-admissions related, 
e.g., in awarding financial aid. 
NASSP Assessment Center 

An assessment center is a psychometric procedure 
that employs multiple activities, including individual 
and group exercises, to develop a behavior profile on 
candidates seeking entry- or middle- level management 
positions. An assessment center is a process that uses 
standardized procedures, particularly simulation 
exercises, to obtain samples of actual behaviors related 
to on-the-job duties. These exercises are good 
simulations of real life occurrences (Landholm, 1986) . 
Trained assessors observe, record, and prepare written 
reports on participants' behaviors in exercises. 
Leaderless group discussions, in-baskets, fact-finding 
activities, and structured interviews are commonly used 
exercises. Assessors pool their observations in 
consensus discussions and rate each participant's level 
of skill on the targeted dimensions. A final report, 
based upon the consensus discussion, is written, and the 
director of the center gives the written and oral 
feedback to each participant. 

The nature of a principal's job comprises the bases 
for the skill dimensions, the nature of the exercises, 
and the expected behaviors observed and rated by the 
assessors. Generalized skills that are important for 
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the principalship comprise the 12 skill dimensions: 
Problem Analysis, Judgment, Organizational Ability, 
Decisiveness, Leadership, Sensitivity, Stress Tolerance, 
Oral Communication, Written Communication, Range of 
Interests, Personal Motivation, and Educational Values 
(Hersey, 1982) . 

Schmitt et al. (1982) found that (a) ratings were 
psychometrically adequate for both internal and content 
validity, (b) the measurements were related to job 
performance and school climate, and (c) the ratings were 
a valid predictor of job success. In a subsequent 
study, Schmitt and Cohen reported that "90 percent of 
the principals with outstanding placement 
recommendations were performing at a 'high' level on the 
job (when rated by their supervisors and teachers) . For 
principals with lower placement recommendations, smaller 
percF.ntages [48%] were performing at a high level" 
(Schmitt & Cohen, 1990, 53, 54). The validation studies 
of Schmitt and his associates lend credence to the 
benefits of obtaining "samples" of job-relat:ed behaviors 
through assessment centers in the selection of 
principals. 

Findings 

The sample for this study consisted of 121 
subjects. All had participated voluntarily in an 
assessment center. Of these 121 subjects, 91 had taken 



MAT/GRE/ Assessment 

12 

the MAT, 33 had taken the GRE, and 3 had taken both 
examinations. 

Pearson Product Moment Correlation Coefficients 
were computed for the scores from the MAT, GRE Verbal 
(GRE-V) , and GRE Quantitative (GRE-Q) with the scores of 
the 12 skill dimensions from NASSP assessment centers. 
No statistically significant correlations existed at the 
.01 level. The correlations ranged from -.05 for GRE 
Quantitative and Personal Motivation to .23 between the 
MAT and Leadership. The relationships between the MAT 
and the GRE Verbal with Leadership, .23 and .21 
respectively, were the strongest for two tests with a 
skill dimension. The correlations between the GRE-V and 
Oral Communication (.22) and between the GRE-V and 
Written Communication (.22) were the strongest for one 
test with two dimensions. 

Not surprisingly, there were extremely small, 
negative correlations in instances where no apparent 
conceptual linkage prevails. For example, between both 
GRE scores, i.e.. Verbal and Quantitative, and the skill 
dimensions of Decisiveness, Range of Interests, and 
Personal Motivation, all the correlations were slightly 
negative; of these negative correlations, the largest 
was only -.05. 

The Miller Analogies Test generally had extremely 
low, positive correlations (.02 with Problem Analysis to 
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.23 with Leadership) with 11 dimensions and one negative 
(-.02) with Educational Values. 

The GRE-Verbal correlations with the skill 
dimensions were extremely low also and were slightly 
negative with four dimensions: Judgment, Decisiveness, 
Range of Interests, and Personal Motivation. The 
highest correlations of the GRE-V were with Written 
Communication (.22), Oral Communication (.22). and 
Leadership ( . 21) . 



Insert Table 1 about here 



The highest correlation of the GRE-Quantitative 
with any skill dimension was with Sensitivity (.16), a 
combination that underscores the separateness of the 
measures, while the GRE-V with Sensitivity was .03. 
Extremely low negative correlations of the GRE-Q were 
with Written Communication (-.01), Decisiveness (-.03), 
Range of interests (-.04), and Personal Motivation 
(-.05) . 

Summary 

Test scores for graduate admissions, as measured by 
the Miller Analogies Test and the Graduate Record 
Examination, had extremely low relationships with 
121 participants' scores in the 12 skill dimensions from 
the NASSP Assessment Center. The low correlations did 
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not link the graduate admissions tests with the NASSP 
assessment center dimensions. Thus the "signs" of 
reasoning and aptitude from the MAT and GRE did not 
necessarily translate into "samples" of job-related 
behaviors . 

Members of admissions committees of departments of 
educational administration should expect little insight 
into applicants' job-related behaviors from admissions 
tests such as the Miller Analogies Test or the Graduate 
Record Examinations. Likewise, hiring officials would 
also have little knowledge of job-related behaviors of 
graduates of administrator preparation programs from MAT 
and GRE admissions test scores although participants who 
get "high" assessment ratings are almost twice as likely 
to get "high" on-the-job ratings as those who get "low" 
ratings in assessment centers. 

The recommendation of the National Policy Board 
that a standardized national test be used to assess 
analytic ability and administrative aptitude is an 
admirable goal. Preparation and validation of such a 
test, comparable to the MCAT or LSAT, will require 
considerable expenditures by test developers. Research 
on the principalship shows the skill areas that are 
prominent in effective school administrators are 
leadership, judgment , sensitivity, organizational 
ability, and problem analysis. The development of a 
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standardized national test to measure such complex 
behaviors would be a significant achievement. Whether 
one test could assess analytic ability and 
administrative aptitude is worth pursuing, - For the 
moment, features of assessment centers that should not 
be overlooked are their validity, objectivity, and 
fairness in the measurement of job-related behaviors. 
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Table 1 

Corrf^lar.ionc; Between Scores Assessment Center Skill 

Dimensions with J:^ Mlll^ Analogies Test 

Graduate Record Examinat ion - - Verbal and Quantitative 



Skill Dimension MAT ORE Verbal ORE 

Quantitative 



Problem Analysis 


.02 


.12 


.07 


Judgment 


.06 


-.05 


.02 


Organizational Ability 


.06 


.13 


.06 


Decisiveness 


.17 


-.05 


-.03 


Leadership 


.23 


.21 


.08 


Sensitivity 


.15 


.03 


.16 


Stress Tolerance 


.06 


.01 


.07 


Oral Communication 


.13 


.22 


.01 


Written Communication 


.17 


.22 


-.01 


Range of Interests 


.15 


-.01 


-.04 


Personal Motivation 


.11 


-.05 


-.05 


Educational Values 


-.02 


.19 


.04 



